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Some Results on Contractive Mappings 


as related to Pattern Recognition 


Abstract 


Several of the presently used techniques in pattern recognition can be 
reformulated as the problem of determining fixed points of a function of the form: 
f(x) * A(x)* X. If Xq is a fixed point of f and if f is contractive at x^, 
then, for any y belonging to a sufficiently small neighborhood of x^, the orbit 
of y will converge to x^. In this paper we develop several general results 
regarding contractive mappings and, in particular, we study functions of the form: 
f(x) = A(x)« X. 

1. Introduction 


Definition Cl.l)* l*et (X, || 11) be a normed linear space and let f : X -»■ X 
be a function. We say f is a contractive mapping of X provided there exists 
a real number a, 0 5 a < 1, such that 1 lf(x) - f (y) 11 s al lx - yl 1 for all 

X, y e X. 

Definition (1.2). Let S be a set and let T : S S be a function. We say 
that Sq € S is a fixed point of T provided T(Sq) - s^. 

If (X, 11 11) is a normed linear space and if f is a contractive mapping 
of X having Xq c X as a fixed point, then x^ is the unique fixed point of 
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f. For, If e 31 are fixed points of f, then 

llxi - X 2 II - llf(x^) - f(x 2 )ll ^ alixj^ - X 2 II . 

Thus, if / X 2 » we have that 1 < a < 1, a contradiction. 

If we impose the added condition that (X, 1 I I I) is complete (that is, a 
Banach space), then any contractive mapping of X has a fixed point. Precisely 
stated, we have the following well-known theorem. 

Theorem (1.3). Let (X, I 11 1) be a Banach space and let f : X -► X be a 
contractive mapping of X. Then, for any z e X, the sequence {f 
converges to a fixed point Xq e X. Moreover, Xq is the unique fixed point 
of f. 


In many cases it may be that f is not contractive on the whole space X 
but rather only at certain points of X. We formalize the concept. 

Definition (1.4). Let (X, !l ||) be a normed linear space and let f : X X 
be a function. We say that f is contractive at x^ e X if and only if there 
exist real numbers € and cx, £>0, Osot<l, such that 

I |f(x) - XqI I S al Ix - XqI I for all x c S^(Xq) » {x e X | | |x - x^| | < c>. We 
say that £ is locally contractive on X provided there exists at least one 
point Xq € X such that f is contractive at x^. 

Observe that if f is contractive at Xq, then 
llf(xQ) - XqII 5 “IIxq " ^ 0 ** “ ’'O ® point of f. The 

following proposition is of prime importance in [2]. 


3 


Proposition (1.5). Lef (X, II II) be a normed linear space and let 
f ; X -*• X be a function. If f Is contractive at e X, with corresponding 
real numbers e and a, then, for any y e S^(Xq), f”(y) e S^(Xq) for each 
positive Integer n, and the sequence if (y)j^^j^ converges to x^. 

Proof . Let y e S^(xq). Then I |f(y) " x^l I s a| |y - x^| | < a» € < e ; 
hence f(y) e S^(Xq). Let k S 1 and suppose that f (y) e 
I lf^(y) - XqI I S a^l ly - XqI I . Then 

l|f*^^(y) - XqII - l|f(f^(y)) - XqM s a||f^(y) - x^^l I s a.a^lly - x^| | 

■ ly - XqI I < e < e . 

IrXI k 

Therefore, f (y) e S^(Xq) and, since a -»• 0, as k -► + *, It follows 
that ||f^(y) - XqII 0, as k -»• + ». Thus, the sequence 
converges to x^ . 

2. Conditions for functions to be locally contractive 

Throughout this section (X, 1 1 I I) will be a finite dimensional real 
normed linear space and f ; X ^ X will be a function. Observe that If X 
Is n-dlmenslonal, then f can be represented as: 



where each f . (x, , . . . , x ) Is a reai-valued function defined on X. We call 
11 n 
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f.(x. ..... X ) the i-th coordinate function of f. We say that f € C' 

11 n ^ ^ 

in a region D (an open connected subset of X) provided each of the coordinate 
functions of f possess continuous first partial derivatives in D. 

The Frechet d erivative of f at the point p ■ (p^^ p^) c X is 

defined to be the matrix of real numbers 



Observe that df(p) determines a linear transformation df(p) : X X 

defined by (df(p))(x) = df(p) • x, where • denotes matrix multiplication. 
When df(p) is looked at in this manner (that is, as a linear transformation 

of X into itself), we call df(p) or df(p) • x the differential of f 
at the point p. Despite its ambiguity, this distinction in terminology should 
cause no difficulty in the sequel. 

Observe that the condition that f is contractive at x^ is equivalent to 
the condition: there exist real numbers 5 and a, 6>0,0sa<l, such 

that llf(xQ + Ax) - f(xQ)|| » II^Cxq ^ OtIlAxM whenever j |Ax| t < 6. 

The following theorem is well-known. (See, for example, [1; Theorem 17, p.264].) 

Theorem (2.1). Let f £ C' in the region D £ X, let p £ D, and let df(p) 
be the Frechet derivative of f at the point p. Then 
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f(p -t Ap) - f(p) + df(p) ' Ap + R(Ap) , 


where 


- 0 . 


Theorem (2.2). Let f c C in the region D £ x and let Xq e D. Then f 

is contractive at Xq if and only if Xq is a fixed point of f and there 

« , , 1 1 df (x_) •Axil 

exist real numbers 6 and 3, 6>0,0S$<1, such that 0 ^ ^ 

I |Ax| 1 

whenever II Axil < 5* 

Proof . ( i — ). Suppose that ^(Xq) “ x^ and that such a 5 and 3 exist. By 
Theorem (2.1), 

II df(Xjj) ’Axil = |lf(xQ + Ax) - £(Xq) - R(Ax)|| > ||f(xQ + Ax) - f(xQ)|| - ilR(Ax)l! 
Thus , for 1 1 Ax 1 1 < 5 , 


1 > 3 2 


df (xg) • Ax I 


lf(x^ + Ax) - f(Xg) 


| R(Ax) 1 1 
llAxIl 


! I f (Xg + Ax) - f (Xq) 


Choose £ > 0 so that 3 + £ “ ot < 1. Since 1 1^| j o j | | j — ^ 


|R(Ax 


there exists Y > 0 such that — r— — n — c whenever 1 1 Ax I I < Y . Thus, 

I I Ax I I I I I 1 1 


for T ■> min{6,Y) » it follows that if llAxM < t , then 


I |f(Xg + Ax) - f (Xg) II S (3 + c) I |Ax| I = a| 1 Ax| 


and hence f is contractive at Xg. 
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( ■ — ^ , Suppose f is contractive at Xq* Then Xq is a fixed point 

of f and there exist real numbers y > 0 and a, 0 £ a < 1, such that 
||f(xQ + Ax) - Xjjll - IlfCXjj + Ax) - fCXp)! I s alUxI! whenever | |Ax| | < Y* 

By Theorem (2.1), 

1 1 df (Xq) • Ax! 1 - Ilf (Xjj + Ax) - f (Xq) - R(Ax) 1 1 i 1 1 f (x^ + Ax) - f (x^) I I + I |R(Ax) j j 

Choose € > 0 so that a + c ■ 3 < 1. Since ^ j |^xM ^ ^ there exists 

X > 0 such thiC ^ I^AxTI ^ ^ ^ ^ whenever | lAxl j < T. Thus, for 6 " min{Y»T}, 

it follows that 

I I df(Xg) . 4x1 I ||f(Xg + 4x) - £(Xg)ll |lRax)ll 

— iTtoTT — " ru5rii iiAxii 


Definition (2.3). Let f € C* in the region D c X, let Xq e D, and suppose 
that df(Xjj) exists. We define the norm of the linear operator df(Xg) to be: 

I Idf(xQ) I i • sup{ I |df (xq) • xII I Mxll - 1, X € X}. 

Theorem (2.4). Under the hypothesis of Theorem (2.2), II df(x^j) l| < 1 if and 
only if there exist real numbers 5 > 0 and 3,0 ^ 8 < 1, such that 
I |df ( xq) • Axl I 

1 iT ~ n ^ 0 whenever II Ax I I < 6 . 


Proof . Suppose ||df(xQ) I I <1. Then 1 > 8 * II^^^C^q^ M 


■ sup 
1 1x1 I 


Ax 


||dt(xQ) * x| I > lldf(xQ) • j j 


I I « I ’ for all Ax. 

llAxll 
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IldfCxQ) # Ax| I 

Conversely, suppose that 1 fdx T l ' s 0 < 1 whenever llAx|| < 6. 

Let X € X, 1 1x1 1 ■ 1, and choose y < ^ • Then 1 lyxl 1 • yl 1x1 1 - y <5 , 
so that 

lldfCx^) .xll - lldf(x^) •^ll - ~ lldfCx^) .yxll 5 Y a * IlYxll - 0 . 

Since X was an arbitrary vector of X of norm 1, it follows that 

lldf(x ) 11 £ 0 < 1 . 

0 

Combining Theorems (2.2) and (2.4), we have the following result. 


Result (2.5). Let f e C in the region D £ X and e D. f is contractive 

at X. if and only if x- is a fixed point of f and lldf(x ) I! <1* 

0 u 0 


Theorem (2.6). Under the hypothesis of Result (2.5), 
11„ ll«(Xo) .4x11 


1 lAxl 1-+0 


lAxl 


exists if and only if 


lim 

1 lAxl 1-K) 


lf(XQ + Ax) - f(xg) 
I lAxl i 


exists. Moreover, if either limit exits 


(and hence the other), they are equal. 
Proof. By Theorem (2.1) 


I I 1 df (x^) - Axil - 11 f (Xq + Ax) - f (Xq) 1 1 I 


- I 1 U(Xq + Ax) - f(Xjj) ‘ R(Ax)| 1 - I |f(xQ + Ax) - f(Xg) 111 
S IU(Xq + Ax) - ^(Xq) - R(Ax) - [f(Xjj + Ax) - f(xQ)]l! 


I lR(Ax)l 1. 
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where j ^ ^ Siven e > 0, there exists 5 > 0 such 

that 


df(Xo> 

lUxI 


Ax 


I I f (X q + Ax) - f (X q) 

)W] 


IR(^) 

llAxll 


< €f 


whenever llAx! I < 6 . Thus, if one limit exists, then the other exists and 
the two limits are equal. 


Lemma (2.7). Uit (X, II ||) be a finite dimensional normed linear space 


and let A ; X -+• X be a linear operator. If 1 1^| * exists, then 

||x| |-K) \ M I ~ * I lA| I , where I |A| I « sup{ I |Ax| I | x e X, | |x| | - l}. 

Proof . Since the unit ball ■ (x c X j | |x| I ■ l) is a cooq)act subspace of 


lAxI 


X and A is continuous:, there exists x, e X such that llx, 11“ 1 and 

MAX II ^ 1 

1 lAx ,1 I “ 1|A||. Therefore, -ri — TT" " llA||. Choose a - ^ for each 
1 1 1 X, I I n n 


positive integer n. Then 


llA(a xJII a^llAxJI 
n 1 n 1 


I lo X. 1 1 

n X 


o^llxJI 
n 1 


|A| I for all n. 


and hence 


11m 


I |A(a X ) I I 
n 1 


I la X 1 1-*-0 I la X I 
n X n X 


|A| I • Thus, if 


lim 


lAxI 


Ixll^ 


exists. 


then 


lim 


Ixl I-*0 


|Ax| 


|A| 


Observe that under the hypothesis of Result (2.5), dfCx^) exists and is 
a linear function of X into itself. Thus, by Result (2.5), Theorem (2.6), and 
Lemma (2.7), we have the following corollary. 
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Corollary .8). Let f e C in the region D £X and let Xq c D. If 

■ L < OB (equivalently* If 


llffl 

IUx||-0 


IldfCx^) • Ax| 


llffl 

llAxll-^0 


Ax 


|f(xQ + Ax) - ffx^) I 




• L < ») , then f la contractive at x. 


If and only If x^ la a fixed point of f and L < 1. 

While It does not seem unreasonable to expect that both Units of Theorem 
(2.6) exist* the following theorem shows that the existence of the Units Imposes 
strong conditions on the linear mapping df(x^). 

Theorem (2.9). Let (X, (*)) be a finite dimensional Inner product space with 
Induced norm ||x!I • /(x*x) and let A ; X X be a linear operator. Then 
A aU for some non-negative real number d and orthogonal matrix U If and only 

llAyl 


if 


exists. 


lIxM-K) ilxll 

Proof . Observe that a linear operator B defined on X Is orthogonal If and 
only If (Bx*Bx) ■ (x,x) * or equivalently, if and only if 1 |Bx| 1 ■ I |x| I for 
all X e X. Note that the theorem Is obviously true If A ■ 0. If k ^ 0 and 


If 


11m 


Ixil-^ 


exists* then, by Lemma (2.7), ■j'}^ 


I |A| I and 


therefore* 


11m 


IxlKO 


Ilxll 


1. Let B 


lAll 


• A and let y e X. 


Choose a ■ — for each positive Integer and consider the sequence of points 
n n 


{o y}* Since I la y| I 0 as n -► + * * 
n n*i n 


then 


liayllH) llayll 

n n 


1 . 
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But 


iB(o^y) 

Tio^TT 


||By|| , , ,, lim 

tlPir lla„yll*oTlvir 


IByll 

llyir 


nierefore, l|By|| - liyil, to that B la orthogonal, and thua, A - IIA||B has 
the required form. 

Converaely, auppoae that A ■ 0(U for aome positive real number 0 ( and 
orthogonal matrix U. Then IlUxll ■ llxll for all x e X, and hence 


11m i lAxI 11m J 

I |x| |-K) I |x! Ilxll-H) 




Corollary (2.10). Let (X, (, )) be a finite dimensional inner product apace 
with Induced norm I ixl I - /(x,x) , let f € C In the region D £ X, and let 


_ 11m 

*0 * Ildxll-H) 


||df(xQ) • Axil 


Ax 


exists, then f la contractive at x. 


If and only If x^ Is a fixed point of f and dfCx^) • aU for some real 
number a, 0 < a < 1, and orthogonal matrix U. 


11m 

lUxll-K) 


We end this section with an example to show that the condition that 

exists Is required In Corollary (2.10). 


Idf(x^) • Ax! 


liAxI 


2 2 2 
Example (2.11). Let E denote real 2-space and let f : E -*> E be defined 

3x X 

by £(x^»x^) - (-— . -| ). Since !lf(x^,X 2 ) - (0,0)11 S ^Wix^^x^) - (0,0)11 

2 

for all (Xj^,x^) c E , and since (0,0) is clearly a fixed point of f, then 
f la contractive at (0,0). However, df(Xj^,X 2 ) at any point (Xj^,X 2 ) € 
la equal to [ J * easily verified that [ ^2 J cannot 

be written In the form aU, where a Is a scalar and U la orthogonal. 
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3. A matrix representation of df (x) 


Let e" denote real or complex n 
defined by f(x) • A(x) • x, where A 

the space of n n matrices. For any 


space and let f 
Is a function from 



£(x) - 



. Thus, for each 1 ■ 1, ...» n, f^ : 


projection of i(x) onto Its ith coordinate. Let A e 
e” (hence f € C* in R). Then, for any point x c 
derivative of f Is 


Let 


df(x) 


A(x) 



3f^(x) 


8f (x) 
n 

~5T 




a (x) 
nn 


; be 

e” Into M , 
nxn 

write 

e” -*■ E^ is the 

C' in the region 
R, the Prechit 
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Therefore) f^(x) * ‘ \ ^ 


.n f^IV!21. 

3xj - W-1 sx^ 


*k ■" ^a 


is the (l,j)th entry of 


df(x). If we denote the matrix 



3Ia^j^(x)] 

3x, 



3[a^k^(x)] 

dx 

n 


*k 


\ 


n 3U„„(x)l 

1^1 3xj^ 


\ 


n 3la„,^(x)] 

k»l 3x 
n 



by B(x), then df(x) = B(x) + A(x). 

Since A : e“ •*• M ^ , it follows that dA(x) can be represented by 

n>^n 

2 

the n X n matrix 
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2 

If we let D(x) be the n x n matrix 



then B(x) * D(x)dA(x) so that df(x) « D(x) • dA(x) + A(x) . 

Although D(x) and dA(x) are not self maps, it Is straightforward 
to show that 

II df(x)|| i I |D(x)dA(x) I I + ||A(x)!I S ||D(x)ll# ||dA(x)|| + | |A(x) | | , 

where we take the usual sup norm on the linear functions df(x), D(x), dA(x), 

and A(x). (For example, llD(x)|| = sup {||D(x)* yl I | I lyl I 2 * ^ ^ » 

^ 2 

where || || and || || „ denote the Euclidean norms in and E*^ , respectively.) 

n 

We next show that llD(x)II ^ I lx| | for all x e e". For, if y e E’^^, 

then 
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and hence, 


llD(x),y!| - 


n-1 / n 

Z I I, x.y, . . 
ir«=0 1 1“1 i^kn + i 


By the Cauchy-Schwarz-Bunyakovskil inequality. 


and hence. 


2 n-l / n \ 

IlDW.yll - ^0 (lil Vkn + ±y 


5o [ (ii ) * (Ji 4 + i')] 

(iSl *1^ kSo 1^1 V + 

(i X ) 




In particular, if | |y| | * 1, we have that | iD(x) • y| 1 I |x| | and hence 

I |D(x) I I » sup { ! |D(x) • y| I I I lyl I - 1, y € £ I |x| i . 

Summarizing the results thus far developed in this section, we have the 


following theorem. 
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Theorem (3.1). Let f : E® -► E™ be defined by f(x) ■ A(x)# x, where 

A is a function from E*' into M . Suppose that A. c C* in the region 

n^ii 

R c (hence f € C* in R). Then, for any point x € R, 

2 

df(x) ■ D(x)* dA(x) + A(x), where D(x) Is the n n matrix 



Moreover, 

11 df(x)ll llD(x)l|. HdA(x)ll + llA(x)ll S lixli# l|dA(x)ll + llA(x)l|. 

It should be noted that the above norm inequality is not sharp enough 
in seeking points at which f is contractive. For, if f is to be contractive 
at Xq, then x^ is a fixed point of f. But then 1 Ix^l I * llf(xQ)lI “ 

- ! |A(Xq) • XqI 1 < I |A(xq) 1 I • i I xqI 1 , so that (if ^ 0) I |A(Xq) 11^1. 
Hence, I Ixq| I • 1 |dA(xQ) I 1 + | |A(Xq) 11^1 and the above inequality cannot 
be used to show that 1 1 df(xQ)ll < 1. 

Finally, we close the paper with some observations relating to the work 
of B. C. Peters and H. F. Walker in [2]. A major result of [2] is that 
a function is locally contractive at a consistent maximum-likelihood 

estimate Xq for sufficiently small values of e > 0. Since ■ x^, 

it suffices to show that |ld$^(xQ)ll < 1. It is easily verified that 
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can be considered as a function from E*^ into itself and that can 

be written in the form; " (1 ** where I is the Identity 

nap on E® and f : e" -»■ e" is defined by f(x) - A(x)# x, where A is 
a function from e" into (That is, f is a function of the type 

considered earlier in this section.) 

Since d$^(x) » (1 - e)I + € df(x) at any point x for which the 

differentials exist, then 

lld$^Cx)ll S (1 - e) + € II df(x)|| 

Thus, if II df(x)| I < 1, then 

1 ld$^(x) II S (1 - £) + cl I df (x) 1 1 5 1 - (1 - 1 1 df (x) I l)c < 1. 

Therefore, since the set of fixed points of f is the set of fixed points 

of for any € > 0, it follows that if f is contractive at y^, then 

is contractive at y^ for any e > 0. 

Conversely, observe that df(x) » — [d$ (x) +(€-!)!], so that 

c c 

II df(x)|| ||d$^(x)|| +- ’A~ - A I . Thus, if ||d<5^(x)il - a, then 

I I df(x) I I < 1 if j (a + |e - 11) < 1. For 0 < e 5 1, ^ (a + |c - 1|) < 1 
is equivalent to a < 2c - 1, and thus, if |Id$^(x)|| < 2e - 1, then 

II df(x)|| < 1. (Note, the assumption a < 2e - 1 implies that 

a < 1 for 0 < e ^ 1.) Thus, if is contractive at y^ and if 

I |d4» (y«) II < 2c - 1, then f is contractive at y*. 

€ U u 

Observe that if c > 1 and if ||d$^(x)|| « a, then 
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II df(x)| I S (ot + € “ 1) • Thus, since the condition ~ (a + € - 1) < 1 

is equivalent to the condition oi < 1, it follows that if lid$^(x)|| < 1, 

then j| df(x)|| < 1. Therefore, for € ^ 1, f is contractive at 

if and only if $ is contractive at y>. We retnark that ?eters and Walker 
€ . 0 

have recently shown that is contractive at a consistent maximum-likelihood 

estimate x^ for 0 < € < 2. 

We suimoarlze these final comments. 

Theorem (3.2). With the notation and hypothesis of the preceding discussion, if 
f is contractive at y^, then is contractive at y^ for all c > 0. 

If 0 < c < 1, I Id^^Cy^)! I < 2e - 1, and y^ is a fixed point of (hence 

is contractive at y^), then f is contractive at yQ. If 1 ^ e, 
f is contractive at y^ if and only if is contractive at y^. 


I 


I 


I 


r 
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